TimeThief: Leveraging Network Variability to Save Datacenter Energy in On-line Data-Intensive Applications

Datacenters running on-line, data-intensive applications (OLDIs) consume significant amounts of energy. However, reducing their energy is challenging due to their tight response time requirements. A key aspect of OLDIs is that each user query goes to all or many of the nodes in the cluster, so that the overall time budget is dictated by the tail of the replies’ latency distribution. Previous work proposes to achieve load-proportional energy by slowing down the computation at lower datacenter loads based directly on response times (i.e., at lower loads, the proposal exploits the average slack in the time budget provisioned for the peak load). In contrast, we propose TimeThief to reduce energy by exploiting the latency slack in the sub-critical replies which arrive before the deadline (e.g., 80% of replies are 3-4x faster than the tail). This slack is present at all loads. While the previous work shifts the leaves’ response time distribution to consume the slack at lower loads, TimeThief reshapes the distribution at all loads by slowing down individual subcritical nodes without increasing missed deadlines. Specifically, TimeThief exploits the slack in the network budget. Further, TimeThief leverages Earliest Deadline First scheduling to largely decouple critical requests from the queuing delays of sub-critical requests which can then be slowed down without hurting critical requests. Using at-scale simulations, we show that without adding to missed deadlines, TimeThief saves 12% and 20% energy at 90% and 30% loading, respectively, in a datacenter with 512 nodes.